Systems and methods for radiologic and photographic imaging of patients

ABSTRACT

Methods for identifying a disease state in a patient and/or for treating a patient having the identified disease state are disclosed and can be based on characteristics identified through machine learning models such as deep learning convolutional neural networks and that are associated with video recordings, audio recordings, infrared images, photographs, and/or radiologic patient images.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/734,255, filed Jan. 3, 2020 and entitled “SYSTEMS AND METHODS FORRADIOGRAPHIC AND PHOTOGRAPHIC IMAGING OF PATIENTS,” which claimspriority to and the benefit of U.S. Provisional Patent Application Ser.No. 62/788,059, filed Jan. 3, 2019 and entitled “SYSTEMS AND METHODS FORRADIOGRAPHIC AND PHOTOGRAPHIC IMAGING OF PATIENTS,” both of which areincorporated herein by this reference in their entirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

The invention was made with government support under the Small BusinessInnovation Research, Phase I Award 1R43TR00229901A1 awarded by theNational Institutes of Health/Center for Advancing TranslationalSciences. The government may have certain rights in the invention.

BACKGROUND Technical Field

This disclosure generally relates to radiologic and photographicimaging. More specifically, the present disclosure relates to systemsand methods for acquiring and combining different forms of patient data,such as radiologic and photographic images, to improve the diagnosis,treatment, and/or capacity for accurately identifying and/or predictingdisease states or other infirmities within a patient.

Related Technology

Advances in computing technology have resulted in a concomitant advancein medical device technologies, including within the field of diagnosticmedicine. Particularly, the past century has demonstrated significantadvances in medical imaging devices. Such advances have been hallmarkedby the improvement and advent of new radiologic devices, such asradiography, computed tomography (CT), magnetic resonance imaging (MRI),and other radiologic imaging systems that allow for the non-invasiveviewing and exploration of internal structures of the body. Thesemedical imaging technologies allow physicians and clinicians to betterdocument, diagnose, and treat pathologies.

Unfortunately, medical imaging studies are prone to mislabeling, such asassociating the wrong demographic information (e.g., a differentpatient's name and medical record number) to a patient's imagingexamination. Patient misidentification errors in medical imaging canresult in serious consequences, such as the misdiagnosis of a diseasestate or the application of an inappropriate treatment regimen.Furthermore, the failure to properly associate a medical image study andpatient identification may propagate to future imaging studies andnegatively affect patient management decisions.

For example, an exemplary wrong-patient error case is illustrated inFIGS. 1A and 1B. The radiograph 100 illustrated in FIG. 1A is aradiograph obtained from a 43-year old black male following coronaryartery bypass grafting and who has a left ventricular assist device andan implantable defibrillator. The patient presented with heart failure,and the healthcare personnel presumed the radiograph 102 illustrated inFIG. 1B to be a radiograph from the same patient captured at an earliertimepoint. However, the radiograph 102 of FIG. 1B is actually aradiograph of a 64-year old white male who had undergone bilateral lungvolume reduction surgery for chronic obstructive pulmonary disease.Radiograph 102 of FIG. 1B was erroneously mislabeled as an earlierradiograph of the patient illustrated in radiograph 100 of FIG. 1 . Whenpresented to seasoned radiologists for diagnosis, 4 out of 4 readersfailed to identify the mislabeling error and assumed that the patienthad suffered a myocardial infarction and complications from surgery inthe interim. In general, the nature of radiologic planar andcross-sectional images makes it difficult to correctly correlateradiologic medical images with patient details absent other identifyingcharacteristics. As in the foregoing example, obvious age and racialdifferences between the patients of the mislabeled radiographs werecompletely lost or unobservable within the radiologic images, making itdifficult for radiologists to identify the error.

A common, acceptable protocol for reducing mislabeling errors involvesthe verification of at least two patient identifiers (e.g., name, dateof birth, social security number, or some hospital registration number)when radiologic images are being obtained. However, such verificationmay not always be possible, including with, for example, many traumapatients, patients who are unconscious or mentally unsound, and infants.Furthermore, even if a technician correctly gathers the requisiteidentifiers, it is difficult to verify with certainty that theidentifiers have been properly associated with the patient. For example,the patient may have been given a mislabeled identification bracelet, orsome other mislabeling error could occur before or after thepoint-of-care verification, that can lead to errors in the canonicaltwo-parameter verification process.

Accordingly, there is a need to minimize or prevent mislabeling ofradiologic images and to properly and consistently identify and/orcorrelate radiologic images with the correct patient, yet these andother disadvantages associated with radiologic imaging remainunaddressed.

BRIEF SUMMARY

Systems, methods, and apparatuses disclosed herein may solve one or moreof the foregoing problems in the art of radiologic imaging. For example,a system for radiographic and photographic imaging of patients isdisclosed herein and includes at least a first camera and a triggerelectronically coupled to the first camera. The trigger can be operableto cause the first camera to capture one or more photographic images ofa patient and can be associated with a manual button of a radiologicdevice that is operable to capture one or more planar or cross-sectionalmedical images of the patient.

In one aspect, the system additionally includes a second camera with thefirst camera being positioned on a first side of the radiologic deviceand the second camera being positioned on a second side of theradiologic device. Alternatively, the first and second cameras can bepositioned on a same side of the radiologic device in a stereoscopic ornon-stereoscopic arrangement. The camera can be coupled to or in closeproximity to the radiologic device or positioned in the same room as theradiologic device.

In one aspect, the trigger causes the camera to capture the imagesubstantially coincident in time with the radiologic device capturingone or more radiologic images.

In one aspect, the one or more cross-sectional medical images includeone or more of an ultrasound image, radiographic image, mammogram, CTimage, scintigraphy image, SPECT image, PET image, or MRI image.

In one aspect, the system includes a microphone and the trigger can befurther configured to cause the camera to capture an image in responseto the microphone receiving a predefined acoustic signal, in particulara tone or operating noise associated with the radiologic device.

In one aspect, the system includes a patient microphone for capturing anaudio recording of the patient. The audio recording can optionallyinclude one or more of an independent or complementary identifier of thepatient or diagnostic information.

In one aspect, the system includes an infrared or thermal imagingcamera. The system can include both an infrared and thermal imagingcamera. Alternatively, the system can include both the camera forcapturing light-based photographs and one of the infrared or thermalimaging camera. In either instance, the cameras can optionally bepositioned stereoscopically relative to one another.

Aspects of the present disclosure additionally include methods of usingthe systems disclosed herein to identify a disease state in a patient,and can include at least the method acts of (i) compiling a datasetincluding a plurality of photographs of a plurality of patients, planarand cross-sectional image data associated with the plurality ofpatients, and one or more diagnoses or disease states associated witheach unique subset of photographs and planar and cross-sectional imagedata for the plurality of patients; (ii) generating a deep learningconvolutional neural network based on the dataset; (iii) identifying aset of predictive characteristics associated with at least some of theone or more diagnoses or disease states based on the deep learningconvolutional neural network; (iv) using one or more aspects of thesystems disclosed herein as to obtain a photograph and cross-sectionalimage data associated with a particular patient; (v) identifyingcharacteristics within the photograph and cross-sectional image datacorresponding to one or more disease states; and (vi) determining adisease state in the particular patient based on the identifiedcharacteristics. In one aspect, determining the disease state in theparticular patient includes the acts of comparing the identifiedcharacteristics within the photograph and cross-sectional image data ofthe particular patient with the set of predictive characteristics andselecting a probable disease state based on the comparison.

In one aspect, the photograph data associated with the particularpatient includes textual patient data on a patient information boardand/or patient monitor, and the method can additionally includeautomatically extracting the textual patient data from the photographdata.

In one aspect, the method additionally includes the act of comparing thephotograph and cross-sectional image data associated with the particularpatient and adjusting the photograph data relative to thecross-sectional image data according to one or more of: scaling a sizeof the photograph, rotating the photograph, or adjusting a noise level,brightness, contrast, color, saturation, or opacity of the photograph.

In one aspect, the method additionally includes the act of administeringa therapeutic regimen to the patient based on the determined diseasestate.

In one aspect, the method additionally includes the act of automaticallydetecting laterality of images of the planar and cross-sectional imagedata based on associated images of the photograph data. In one aspect,the method additionally includes comparing the detected laterality ofimages of the planar and cross-sectional image data to manually enteredlaterality data, and automatically flagging images having a mismatchbetween detected laterality and manually entered laterality.

Embodiments of the present disclosure can additionally include methodsfor automatically detecting a misidentified study and can include atleast the steps of receiving and comparing photographs of a patienttaken at a first and second timepoint and determining similaritiesbetween a subject depicted in the photographs based on patientrepresentations obtained from a neural network model, or other facialrecognition algorithm, trained using patient photographs and/or trainedusing general facial photographs. In one aspect, the patientrepresentations are obtained from a neural network model trained usingpatient photographs. In another aspect, the patient representations areobtained from a facial recognition algorithm trained using generalfacial photographs. In one aspect, the detecting a misidentified studyincludes detecting a similarity value between the compared photographsthat is less than a threshold similarity value, and the methodadditionally includes generating and sending an alert identifying thedetected misidentified study.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to describe the manner in which the above recited and otheradvantages and features of the disclosure can be obtained, a moreparticular description of the disclosure briefly described above will berendered by reference to specific embodiments thereof, which areillustrated in the appended drawings. It is appreciated that thesedrawings depict only typical embodiments of the disclosure and are nottherefore to be considered to be limiting of its scope. The disclosurewill be described and explained with additional specificity and detailthrough the use of the accompanying drawings in which:

FIG. 1A and FIG. 1B each illustrate radiographs within a simulatedwrong-patient error case;

FIG. 2A and FIG. 2B respectively illustrate the radiographs of FIG. 1Aand FIG. 1B along with redacted photographs of each correspondingpatient taken at the time of the radiograph, the photographs being shownin association with the respective radiograph for each patient;

FIG. 3 illustrates an exemplary system having a camera system associatedwith a radiographic imaging system, in accordance with one or moreembodiments of the present disclosure;

FIG. 4 illustrates an exemplary trigger system associated with a camerasystem and other computing system in electronic communication with thecamera system, in accordance with one or more embodiments of the presentdisclosure;

FIG. 5 illustrates an exemplary trigger unit, in accordance with one ormore embodiments of the present disclosure;

FIG. 6A illustrates an exemplary digital microphone for capturing audiodata and/or for use as a trigger unit, in accordance with one or moreembodiments of the present disclosure;

FIG. 6B illustrates a spectrogram of an alert tone recorded during X-rayacquisition from a portable X-ray unit, the spectrogram showing two puretones lasting roughly 0.25 second being visible as having more powerthan surrounding audio (including talking), in accordance with one ormore embodiments of the present disclosure; and

FIGS. 7A-7D illustrate an exemplary co-registration of photographic andradiologic images where a patient photograph (FIG. 7A) can beco-registration with a radiologic image (FIG. 7B) such that non-anatomicartifacts can be identified and highlighted within the radiologic image(as shown in FIGS. 7C and 7D), in accordance with one or moreembodiments of the present disclosure.

DETAILED DESCRIPTION

Before describing various embodiments of the present disclosure indetail, it is to be understood that this disclosure is not limited tothe parameters of the particularly exemplified systems, methods,apparatus, products, processes, and/or kits, which may, of course, vary.Thus, while certain embodiments of the present disclosure will bedescribed in detail, with reference to specific configurations,parameters, components, elements, etc., the descriptions areillustrative and are not to be construed as limiting the scope of theclaimed invention. In addition, the terminology used herein is for thepurpose of describing the embodiments and is not necessarily intended tolimit the scope of the claimed invention.

Overview and Exemplary Advantages of Disclosed Imaging Systems

As discussed above, wrong-patient errors in radiology—where onepatient's imaging examination can be erroneously filed in anotherpatient's record in the radiology picture archiving and communicationssystem (PACS)—can lead to devastating consequences for both patientsinvolved, as well as potential for healthcare provider liability formisdiagnosis and/or incorrect treatment. Embodiments of the systems andmethods disclosed herein address at least some of the foregoingproblems. For example, an imaging system can include a camera forpoint-of-care acquisition of patient photographs that can be associatedwith the radiologic medical image taken at the same time. These photosserve as intrinsic, externally visible, biometric identifiers, alongwith medical imaging studies. Embodiments of the present disclosure mayadditionally ensure that the photos are transmitted without any humanintervention to the requisite PACS.

By leveraging a combination of timestamps and machine identifiers (e.g.,an identifier associated with a particular radiographic imaging system),embodiments of the present disclosure may allow the photos to be addedto the correct patients' imaging records within PACS. Additionally,implementing the disclosed systems in a clinical setting has clearlydemonstrated that the presence of photographs significantly increasesthe detection rate of wrong-patient errors during interpretation ofradiographs, specifically, and which may extend to radiologic images,generally. Surprisingly, the inventors have also shown that radiologistsunexpectedly demonstrated a significant decrease in the amount ofinterpretation time when photographs were provided with radiographs ofthe corresponding patient.

For example, regarding the wrong-patient error case discussed above(illustrated in FIGS. 1A and 1B), there is a reduced or eliminatedwrong-patient error when implementing embodiments of the presentdisclosure. As shown in FIGS. 2A and 2B, a resultant combined andcomparative image set can be seen where the radiograph 100 is associatedwith a photograph 104 of the patient taken at the same time as theradiograph 100 was captured. Similarly, radiograph 102 is properlyassociated with a photograph 106 taken of the patient at the same timeas the radiograph 102 was captured and is not assumed to be an updatedradiograph of radiograph 100. The photographs 104, 106 clearlyillustrate that the radiographs 100, 102 are not from the same patient,and when presented to seasoned radiologists for diagnosis, 4 out of 4readers were easily able to identify the simulated wrong-patient errorinstead of making an erroneous diagnosis.

Among other benefits, associating a photograph of the patient along withthe captured radiologic image can reduce wrong-patient error byproviding a set of easily verifiable identifying characteristics for thepatient. As in the foregoing example, the obvious age and racialdifferences between the patients of the mislabeled radiographs areapparent given even a cursory observation. Such easily observabledifferences can extend to gender, body type, hair color, and otherphysical differences. Because the photograph can serve as identifyingcharacteristics of the patient, it further enables misidentifiedradiographs to be associated with the correct patient. This isunprecedented since, in the past, misfiled radiologic images wereusually discarded, implying that a patient had to undergo repeat imagingwith its attendant radiation. Embodiments of the present disclosure can,therefore, reduce or eliminate the reimaging of patients whoseradiologic images have been misfiled. This allows for extended utilityof the radiologic device (as it can now be used for other non-repeatedstudies) and reduces patient exposure to potentially harmful radiation.

Additionally, associated photographs obtained by embodiments of thepresent disclosure can also provide image-related clinical context thatpositively impact diagnosis. In some examples, soft tissuesabnormalities, which are not visible on the radiographs but are visibleon the photographs, can be helpful in focusing the radiographicinterpretation of the underlying bone. In other examples, the wide-anglelens used by some disclosed camera systems can capture the patientmonitors in the hospital room, which provide more information to theradiologist—information that would have taken the interpretingradiologist several steps of wading through the patient's electronicmedical record to find. In some instances, facial features of patientswho have suffered acute strokes may clearly point to the underlyingdiagnosis. In other instances, the photographs may clearly show whetherthe patient was upright or lying flat—information that was not visibleon the corresponding radiologic image. This positioning information canbe critical to the interpreting radiologist, for example, when excludingthe presence of free air in the abdomen, which is an ominous sign thatcan suggest perforated bowel and can dramatically increase theefficiency and confidence of radiologic image interpretation.

Additionally, as described in more detail below, embodiments of thepresent disclosure can enable the rapid identification of variousnon-anatomic features present in radiologic images (e.g., feeding tubes,endotracheal tubes, wires for monitoring patient vitals, or other tubesand wires), which can allow the radiologist to more efficientlydiscriminate between important and non-important features in radiologicimages. Such non-anatomic features can also provide useful informationof the type of care that is being or has been provided to the patientand/or provide additional indications as to the physical condition ofthe patient.

The embodiments disclosed herein provide radiologists with additionalpatient identification in the form of a photograph (e.g., of the faceand chest obtained at the point-of-care with portable radiography). Theinventors have demonstrated that systems disclosed herein that include acamera system for capturing a patient's photograph prior to and/orduring radiography improved radiologists' ability to detectwrong-patient errors, in some instances, by about 5-fold. Surprisingly,adding the patient's photo to the radiological image was also shown toreduce the radiologist's interpretation time of the correspondingradiographs. Thus, in addition to decreasing wrong-patient errors, thesystems and methods disclosed herein beneficially enable an increaseinterpretation efficiency, thereby decreasing interpreting physiciancost—a substantial portion of the cost of an imaging study. In addition,patient photographs can increase empathy and satisfaction amonginterpreting radiologists, resulting in improved patient-centeredradiology care.

Additional features and advantages of the disclosure will be set forthin the description that follows and will be obvious, at least in part,from the description or may be learned by the practice of thedisclosure. The features and advantages of the disclosure may berealized and obtained by means of the instruments and combinationsparticularly pointed out in the appended claims. These and otherfeatures of the present disclosure will become more fully apparent fromthe following description and appended claims or may be learned by thepractice of the disclosure as set forth hereinafter.

Exemplary Systems for Radiographic and Photographic Imaging

As provided herein, embodiments of the present disclosure can includesystems and apparatuses for radiographic and photographic imaging ofpatients. For example, as shown in FIG. 3 , a medical imaging andarchiving system 110 for radiographic and photographic imaging ofpatients can include a radiographic imaging system 112 and a camerasystem 114 operable to generate medical images and photographic imagesof a patient, respectively, to prevent and/or reduce the occurrenceand/or likelihood of patient misidentification errors, among otherbenefits disclosed herein.

In accordance with embodiments of the present disclosure, medical imagesmay be generated at the system 110 by one or more medical imagingmodalities, including any known medical imaging modality or combinationof modalities, and may be generated by any known medical imaging deviceor system. For example, the medical imaging modality used by the system110 may include, but is not necessarily limited to: ultrasound,radiography, mammography, computed tomography (CT), computed tomographyperfusion (CTP), scintigraphy, single-photon emission computedtomography (SPECT), positron-emission tomography (PET), and magneticresonance imaging (MM).

As shown in FIG. 3 , the imaging modality can be implemented by aradiographic imaging system 112 configured to generate medical imagesand may include a radiologic device that generates one or more planar orcross-sectional images (i.e., radiologic images 125) in the form of anultrasound image, radiographic image, mammogram, CT image, CTP image,scintigraphy image, SPECT image, PET image, or MM image, respectively.With continued reference to FIG. 3 , a radiographic imaging system 112can include one or more of a CT scanner 116, a digital radiography (DR)machine 118, a mammography system 120, an Mill scanner 122, or otherimaging modality system 124 known in the art (e.g., an ultrasounddevice, a stationary X-ray machine, a scintigraphy device such as ascintillation camera, a SPECT scanner, a PET scanner, or similar)—any ofwhich may be coupled to an image processing, computing, and/or storagesystem, such as PACS or other component of radiologic system 126.

The medical imaging and archiving system 110 can additionally include acamera system 114 operable to take photographs of the patient or subjectbeing imaged by the radiographic imaging system 112 and a trigger 128 inelectronic communication with the camera system 114. In someembodiments, the camera system 114 and trigger 128 can be controlled byone or more components of the camera system 114 and/or radiographicimaging system 112. This can include, for example, controlling thetiming and number of pictures captured by these systems 112, 114.

In some embodiments, the camera is coupled directly to the radiologicdevice, or it may be positioned in the same room as, or in the vicinityof, the radiologic device. For example, the camera system can include animage sensor or camera 130 associated with and/or operated by acomputing system. As shown in FIG. 3 , the computing system associatedwith the image sensor or camera 130 can include one or more processors132 and memory (e.g., a hardware storage device 134). In someembodiments, the camera can be operated by a single-board computer, suchas a Raspberry Pi or similar compact computing system, and can beconfigured to communicate with peripheral devices and/or a networkthrough the onboard (or associated) networking device 136. The camerasystem 114 can additionally process and temporarily (or permanently)store the captured images. In some embodiments, however, the images 135captured by the camera system 114 can be transmitted (e.g., via a wiredor wireless signal through networking device 136) to a storage silo 138or integration system, which in some embodiments may be the same systemor subsystem, where the photographs are processed and/or temporarilystored. In some embodiments, the images 135 captured by the camerasystem 114 can be processed and temporarily stored before beingtransferred to and stored within an archive, such as PACS or othercomponent of radiologic system 126.

The photos captured at the camera system 114 serve as intrinsic,externally visible, biometric identifiers, along with medical imagingstudies. After being captured at the camera system, the images 135 aretimestamped and associated with a machine identifier that indicateswhich radiographic imaging system was used to capture an analogousmedical image of the patient. This can be performed, for example, at thecamera system 114 (e.g., by executing, at the processor(s) 132,computer-executable instructions stored on the hardware storage device134 that configure the camera system to timestamp and/or update themetadata associated with the captured image to include a timestamp andthe machine identifier associated with the image). An exemplary methodflow of such is provided in FIG. 4 .

Alternatively, the images can be transferred to an image modificationmodule 140 configured to tag the images with identifying information,such as a timestamp and machine identifier. In some embodiments, theimage modification module 140 is associated with the storage silo 138,as shown in FIG. 3 or may alternatively be associated with the camerasystem (not shown). The image modification module 140 can beneficiallyensure that the images are properly identifiable such that, whentransferred to PACS or other component of radiologic system 126, theimages are added to the correct patient's imaging records within PACS orother component of radiologic system 126.

In some embodiments, the trigger 128 or switch associated with themedical imaging and archiving system 110 can be associated with a manualbutton on a controller configured to capture one or more cross-sectionalimages of the patient. This can include, for example, integrating thetrigger into a manual controller associated with the radiographicimaging system 112 such that operating the manual controller to capturea radiologic image of the patient using the radiographic imaging system112 activates the trigger and causes the camera to capture a photographof the patient and optionally the patient's surroundings.

Accordingly, activating the camera can cause the camera to capture aphotograph of the patient substantially coincident with capturing one ormore radiologic images from the radiologic device. Additionally, oralternatively, activating the camera can cause the camera to capture thephotograph prior to or after capturing the radiologic images.

It should be appreciated that the patient image may be a portrait of thepatient or an image of any portion of a patient's body. The patientimage may be confined to the face of a patient, from about the foreheadto the chin, or may include additional portions of a patient's body,such as the upper chest area of the patient. Additionally, oralternatively, the patient image may include the area of the patient forwhich the medical image is obtained. It will be appreciated thatmultiple photographs showing different patient views can be captured.For example, the system can be configured to image the patient prior toand/or after imaging with the radiographic imaging system, such as whenthe patient is being prepared for or removed from a CT or MRI scanner.In some instances, the photograph of the patient is taken at the sametime or substantially the same time as the radiograph is being taken.This can beneficially allow the radiologist to match the body positionbetween the cross-sectional medical image and the photograph as a fastand efficient confirmation that there likely has not been a patientmisidentification or laterality errors.

In some embodiments, the patient image may be still or dynamic. Theselection of a still or dynamic image may be controlled by the user orbe automatic. Accordingly, the patient image may be a digital photographor a digital video, and the selection or determination of a still ordynamic image capture may be preset or selectable by the imagingtechnician, a physician, or another healthcare provider. The patientimage may be captured and/or stored in any known digital format. Forexample, the format may be chosen from any of JPEG, TIFF, PNG, GIF, BMP,WAV, AVI, FLV, MJ2, MPEG, MPEG-2, MP4, WMV, ASF, or QTFF. In someembodiments, the file format may be selected according to standards fortransmitting radiological or other medical information, such as DICOMformat, although if the patient image is obtained in a format other thanthe format of the medical record storage system, such as DICOM, thepatient image may be later converted.

Furthermore, the format of the patient image may depend on the medicalimaging device configuration. For example, a patient imaging deviceconfigured to obtain a still or dynamic patient image may be integratedwith the medical imaging device. Alternatively, a patient imaging deviceconfigured to obtain a still or dynamic image of the patient may be aperipheral device. In either case, the patient image may be obtained bya digital camera operable to capture photographs and/or video. As anon-limiting example of the foregoing, the patient image may be obtainedby a charge-coupled device or a complementary metal oxide semiconductor(CMOS) camera. The camera may be operable to capture images within thevisible or infrared light spectra or may capture thermal images. In someembodiments, the infrared and/or thermal imaging camera is separate fromthe visible light camera and may be positioned relative theretostereoscopically, as discussed in more detail below.

In some embodiments, the camera system includes a camera having awide-angle lens. The use of wide-angle lenses beneficially allows forthe acquisition of potentially relevant information in the patient'sroom. For example, if there is a whiteboard in the patient's room behindthe patient with information such as phone number, patient location(room), physician name, nurse name, etc., this information can be usedby the interpreting doctor (e.g., radiologist) to contact theappropriate physician or nurse without having to look up thisinformation elsewhere. Additionally, if the patient monitor, which hasinformation such as heart rate, blood pressure, respiratory rate, pulseoximetry, etc., is placed within the field of the wide-angle lens, thenthis information can also be seen while interpreting the radiographs andcan help inform the interpretation. It should be appreciated that insome embodiments, automated image analysis can be performed on capturedphotographs to extract contextual information from the patient'senvironment and transmit it to the electronic medical record separately.

The use of different photographic imaging modalities, such as visiblelight photography, infrared photography, and/or thermal photography canserve multiple beneficial purposes. For example, in low-light scenarios,photographs taken in the visible light spectrum may have too much noisein the darker portions of the photograph whereas infrared photographymay beneficially be better suited to capture a higher fidelity photo ofthe patient for identification, verification, and/or diagnosticpurposes. Additionally, thermal imaging can be incorporated into camerasystems disclosed herein as a beneficial way to gather additional,potentially useful information. For example, thermal imaging can be usedto identify “hot spots” on the patient's body, which would indicatelocalized inflammation and/or infection that can be correlated with themedical images captured by the radiographic imaging system.

In some embodiments, low-light performance can be improved by includinga camera without an infrared (IR) filter and with LED IR illumination.The IR LEDs cast a light that is invisible to the naked eye but isvisible to an IR filterless camera. Because it is preferable to have animage with IR removed when possible, a dual camera solution can beimplemented for portable X-ray and other low-light applications. Forexample, when the overall brightness of the normal image is above athreshold, the visible light image may be used, whereas, when thebrightness is below that threshold, the IR image may be used.

Additionally, or alternatively, the imaging systems disclosed herein caninclude additional camera systems 142 that are each operable to captureimages of the patient (e.g., at different positions or angles and/or indifferent imaging modalities). In some embodiments, multiple cameras orcamera systems can provide significant additional benefits. For example,for a mammography system, the patient's face may be on one side of themachine while the opposing breast is imaged. In this situation, twocameras may be particularly useful—one on each side of the machine. Ifeach camera is configured to image the patient at the same time thebreast is being imaged, at least one of the cameras will capture aphotograph of the patient's face. Knowing which camera captured thepatient's face can inform which breast was being imaged. To assist inthis, each photograph may be tagged with metadata identifying the cameraor camera location used to capture the image, which can then be used toinfer or determine laterality. Accordingly, multiple cameras canbeneficially enable documenting of laterality and avoid or reducelaterality errors (e.g., errors documenting left versus right side ofpatient). Additionally, in some embodiments, the system is configured toautomatically detect laterality (e.g., left, right) and/or position(e.g., face up, face down, feetfirst, headfirst). The detectedinformation can, in some embodiments, be automatically added to theimage header and/or PACS (or other hospital system), and thisinformation may, in some instances, be used to automatically compareagainst what was entered by the technologist and raise an alert if thereis a discrepancy between the two. This can beneficially reduce potentialmisidentification errors and act as another layer of verification andsafety. It should be appreciated that the determination of lateralityand/or position can be automatically determined following supervisedtraining of an artificial intelligence engine, such as the artificialintelligence engine 146 of the integration system in FIG. 3 .

In some embodiments, the integration system, which can include anartificial intelligence engine 146, and/or the image modification module140 can be independent of the camera system 114 and could be used withphotographs already existing in the radiologic system 148. Additionally,in some embodiments, direct integration between the radiologic modalityand photographs obtained by the camera system 114 or third party systemand the integration system (or components thereof, such as PACS) can beimplemented.

In another exemplary embodiment, a camera can be positioned on each sideof the CT gantry, which can increase the likelihood that the patient'sface is captured regardless of whether the patient goes into the gantryheadfirst or feet first. In some embodiments, one camera may be used andpositioned on the side of the gantry associated with the patientposition found in the majority of CT images. In some embodiments, thecamera is on a track and can automatically identify a patientorientation and position itself along the track to place the patient'sface within the viewing area of the camera. This can include a lineartrack along a longitudinal axis of the gantry (e.g., to capture aphotograph of the patient's face in a feet-first or head-firstorientation) and/or an orbital track around the gantry (e.g., to capturea photograph of the patient's face in a prone or supine orientation).

As discussed above, the trigger 128 can be a manual trigger ormechanical switch responsive to the activation of the radiographicimaging system by the technician or attending healthcare provider (e.g.,as shown in FIG. 5 ). In some embodiments, the trigger may be operableto automatically capture the photograph for optimal performance andreliability.

For example, one method to automatically trigger photo acquisitionincludes incorporating a split-core current sensor as a switch andthereby be completely decoupled from the technologist. The split-corevariety of a current sensor can be non-intrusively clipped around thecable supplying current to the X-ray filament. While resulting tubecurrents are in the mA range, the supply current is high enough to besensed by sensors (e.g., HCT Series Current Transducer). The sensor maygenerate either a detectable voltage or current that may be used totrigger the camera system to acquire a photo of the patient when theX-ray tube is emitting. In some embodiments, the current sensor can bepowered by the camera system. As a result, a separate step-up circuit,and potentially a battery power supply if needed, may be implemented. Itshould be appreciated that in some embodiments, the current-sensingtrigger can be configured to be tunable to different sensed amperages,allowing the same switch design to be used for different radiologymachines. One advantage of this approach includes potentially increasedreliability as other automatic triggers disclosed herein can be moreresource intensive and/or have signal processing steps that occur at thecamera system.

As an additional example, and with continued reference to FIG. 3 , acamera system 114 can include a microphone 144 (or other transducer)operable to automatically trigger photograph acquisition by the camerasystem 114. One such exemplary embodiment of a microphone is shown inFIG. 6A. In some embodiments, the microphone acts as a trigger, causingthe camera to capture a photograph in response to the microphonereceiving a predefined acoustic signal.

The predefined acoustic signal can be any acoustic signal. For example,the predefined acoustic signal can include a tone or operating noiseassociated with a radiologic device. In an exemplary embodiment, asillustrated in FIG. 6B, the predefined acoustic signal can include analert tone recorded during X-ray acquisition from a portable X-ray unit.FIG. 6B illustrates a spectrogram of such an exemplary alert tone. Thespectrogram shows two pure tones lasting roughly 0.25 sec visible ashaving more power than surrounding audio (including talking).

Upon detecting this tone, the camera system may take a photograph of thepatient. Audio data may be buffered and its frequency content analyzedfor the presence of the tone. In some embodiments, a module may be addedto the software framework of the imaging system to train and incorporatea library of detectors for known radiology machine models. With thissoftware structure, the detector can be modified for each model ofradiography unit. The advantages of this triggering approach include theease of integration, adaptability, and the dual-use of the microphone,which may also be used to record the patient's voice to provide furtherpatient identification and clinical information.

In some embodiments, the camera can capture an image or a series oftime-lapsed or dynamic images, such as a video recording. Video data canbeneficially provide additional correlated facial images that canenhance facial recognition algorithms. A brief video clip providing thepatient's chief complaint directly from the patient embedded with theradiographs can also beneficially speed up the interpretation and add toradiologists' confidence in the interpretation.

Additionally, or alternatively, an audio recording of the patient can becaptured. This audio can be used to further serve as an independent orcomplementary identifier of the patient. For example, a patient may beasked to state their name while or just before the radiologic images areobtained. This audio can also be used to obtain, for example, surveyinformation from patients. In some embodiments, the survey informationcan be uniquely tied to the radiologic imaging study and can be used,for example, as an analytic tool to improve service performance withinthe imaging department.

In some embodiments, the audio can be used as diagnostic information.For example, slurring of speech can be indicative of a stroke, and thisinformation can be combined with the photographic or radiologic imagingto improve diagnosis. Similarly, video recordings can reveal heart rateinformation since subtle rhythmic facial hue changes with the beatingheart can be detected by advanced image processing. As another example,patients can answer simple questions about their chief complaint andthese answers can be stored along with the imaging examination. Theseanswers may provide interpreting radiologists with the criticalinformation regarding the reason (indication) for the examination andthus impact the interpretation. Quite often the reasons for theexamination given on the requisition, i.e., order, for the imaging studyare vague since they may be generic summaries based on billing codes,rather than clinically relevant information.

Audio and/or video can also be used as a record of the informed consentfor the radiologic procedure. Video and audio content may be leveragedto improve and/or supplement both facial recognition and clinicalcontext provided to the radiologist. In addition, text of radiology examspeech may be stored within the hospital information system.

Methods for Standardizing Image Storage and Display

Photographs captured using the foregoing imaging systems can beincorporated into methods for standardizing image storage and display.As a non-limiting example, an exemplary method can include the methodacts of (i) acquiring or receiving a photograph of a patient from acamera or storage silo containing said photograph, (ii) acquiring orreceiving one or more planar or cross-sectional images of the patientfrom a radiologic device or storage silo containing said cross-sectionalimages, (iii) comparing the photograph and the one or morecross-sectional images; and (iv) adjusting the photograph relative tothe one or more cross-sectional images. In some embodiments, acquiringthe photograph includes capturing the photograph using any of thesystems for radiographic and photographic imaging of patients disclosedherein. Additionally, adjusting the photograph relative to thecross-sectional images can include scaling a size of the photograph,rotating the photograph, adjusting a brightness, contrast, color,saturation, or opacity of the photograph, or combinations thereof andmay be implemented using a machine learning model or other facialrecognition algorithm.

For example, autorotation performance may be improved by training aconvolutional neural network (CNN) to output the correct rotation givena patient photo of unknown rotation. In one embodiment, autorotation canbe accomplished by performing CNN face detection on each of the fourpossible rotations. If a face is detected in more than one of therotations, the rotation with the largest response is assumed correct.There are two downsides to this approach: (1) it can be relatively timeconsuming, taking about 20 seconds per photograph, and (2) it relies onface detection, which can be difficult to detect in extremely low-lightconditions or when a medical apparatus is obscuring the face of thepatient. By training an autorotator specifically for autorotation,calculation time can be decreased while also increasing accuracy.

As an additional example, methods can include identifying and/orverifying a patient's identity based on photographs taken of the patientat multiple timepoints and/or within multiple frames of a video. Suchmethods can automatically eliminate or flag mislabeled or mismatchedimages thereby reducing wrong-patient errors.

Systems for Automated Registration of Corresponding Photographs andRadiology Images

A system that automatically registers and aligns correspondingphotographs and radiographs is needed for proper clinical display andalso to leverage the mutual information present in both for machinelearning and computer vision applications.

Accurate multi-modal image registration of the larger patient photographaround a simultaneously acquired radiograph relies primarily on threecharacteristics: rotation, translation, and scale. Skew may not beconsidered because the radiograph and photograph have the same imageplane, i.e., they are pointed in roughly the same direction. Embodimentsof the present disclosure include systems and methods for automatedregistration of corresponding photographs and radiology images. In oneembodiment, multi-modal image registration techniques are used tomaximize the mutual information between two given images of differentmodalities and thereby allow for the identification of rotation,translation, and scale between photographs and radiographs. For example,as shown in FIGS. 7A and 7B, a radiograph of the patient is rotated,translated, and scaled over a photograph of the patient such that thephotograph and radiology image are co-registered.

A system that automatically indicates external, non-anatomic featurespresent in radiology images, such as lines and tubes, may allow theradiologist to more efficiently discriminate between important andnonimportant features in radiology images. In some embodiments, systemsand methods disclosed herein further enable the automatic annotation ofexternal features in radiology images. An exemplary method includes theact of locating external features common to both the photograph andradiograph. This may be accomplished either by region-based correlationtechniques (regions of high correlation correspond to common features)or as regions of large mutual information. As shown in FIG. 7C, theexternal features common to both the photograph and radiograph arelocated.

After being located, the presentation of these annotated externalfeatures is refined by improving the color, opacity, size, etc. tomaximize the utility of these annotations. As a result, photographs maybe spatially registered to the corresponding radiographs, allowing foraccurate display and increasing the future usefulness of point-of-carephotographs in informatics and machine learning contexts. The locationof external features such as lines and tubes may be automaticallyannotated and recorded in a manner conducive to conveying locationinformation to radiologists directly on radiographs.

Methods for Identifying a Disease State in a Patient

The systems disclosed herein may additionally be implemented withinmethods for identifying a disease state in a patient. Particularly, acombination of photographic and radiographic characteristics may beidentified as correlating with a disease state, and thesecharacteristics may be elucidated using a machine learning model andapplied to identify, predict, and/or treat patients by administering a(prophylactic or reactionary) therapeutic regimen based on theidentified/predicted disease state.

For example, a method for identifying a disease state in a patient caninclude the method acts of (i) acquiring a photograph andcross-sectional image data associated with a patient, (ii) identifyingcharacteristics within the photograph and cross-sectional image datacorresponding to one or more disease states, and (iii) determining adisease state in the patient based on the identified characteristics.Methods may additionally include acts of (iv) compiling a dataset, (v)generating a deep learning convolutional neural network based on thedataset, and (vi) identifying a set of predictive characteristicsassociated with at least some of the diagnoses or disease states, basedon the deep learning convolutional neural network. The dataset compiledin act (iv) above can include, for example and among other things, aplurality of photographs of a plurality of patients, planar andcross-sectional image data associated with the plurality of patients,and one or more diagnoses or disease states associated with each uniquesubset of photographs and cross-sectional image data for the pluralityof patients.

In some embodiments, determining the disease state in the patient caninclude comparing the identified characteristics within the photographand planar or cross-sectional image data with the set of predictivecharacteristics and selecting a probable disease state based on thecomparison. Additionally, or alternatively, the methods can includeadministering a therapeutic regimen to the patient based on thedetermined disease state.

Additionally, in some embodiments, patient photographs can be used toallow for an automated method of visually assessing patient well-being.The automated assessment can, for example, provide a quantitativemeasure of how “healthy” the patient looks. These measures can becorrelated with the medical imaging studies over time with deep learningso that the appearance of the patient can potentially provide adiagnosis or diagnostic clues.

Abbreviated List of Defined Terms

To assist in understanding the scope and content of the foregoing andforthcoming written description and appended claims, a select few termsare defined directly below.

The term “healthcare provider” as used herein generally refers to anylicensed and/or trained person prescribing, administering, or overseeingthe diagnosis and/or treatment of a patient or who otherwise tends tothe wellness of a patient. This term may, when contextually appropriate,include any licensed medical professional, such as a physician (e.g.,Medical Doctor, Doctor of Osteopathic Medicine, etc.), a physician'sassistant, a nurse, a radiology technician, a dentist, a chiropractor,etc. and includes any physician specializing in a relevant field (e.g.,radiology).

The term “patient” generally refers to any animal, for example a mammal,under the care of a healthcare provider, as that term is defined herein,with particular reference to humans under the care of a primary carephysician, oncologist, surgeon, or other relevant medical professional.For the purpose of the present application, a “patient” may beinterchangeable with an “individual” or “person.” In some embodiments,the individual is a human patient.

As used herein, the term “patient information board” refers to anenvironmental object in the vicinity of the patient and which hasassociated therewith patient information (e.g., name, date of birth,patient ID, medications, allergies, medical history, vitals, etc.) orother information relevant to the patient's treatment, diagnosis, ormedical history, such as the name and/or contact information ofhealthcare providers tending to the patient, symptoms, prior illnessesor procedures, or similar. It should be appreciated that the term“patient monitor” can include any of the foregoing informationassociated with a patient information board but provided on anelectronic display or device, such as a vital monitoring system, tablet,or similar.

The term “physician” as used herein generally refers to a medicaldoctor, and particularly a specialized medical doctor, such as aradiologist, oncologist, surgeon, primary care physician, or otherspecialized medical doctor interpreting or viewing radiologic images.This term may, when contextually appropriate, include any other medicalprofessional, including any licensed medical professional or otherhealthcare provider, such as a physician's assistant, a nurse, aveterinarian (such as, for example, when the patient is a non-humananimal), etc.

Computer Systems of the Present Disclosure

It will be appreciated that computer systems are increasingly taking awide variety of forms. In this description and in the claims, the term“computer system” or “computing system” is defined broadly as includingany device or system—or combination thereof—that includes at least onephysical and tangible processor and a physical and tangible memorycapable of having thereon computer-executable instructions that may beexecuted by a processor. By way of example, not limitation, the term“computer system” or “computing system,” as used herein is intended toinclude personal computers, desktop computers, laptop computers,tablets, hand-held devices (e.g., mobile telephones, PDAs, pagers),microprocessor-based or programmable consumer electronics,minicomputers, mainframe computers, multi-processor systems, networkPCs, distributed computing systems, datacenters, message processors,routers, switches, and even devices that conventionally have not beenconsidered a computing system, such as wearables (e.g., glasses).

The memory may take any form and may depend on the nature and form ofthe computing system. The memory can be physical system memory, whichincludes volatile memory, non-volatile memory, or some combination ofthe two. The term “memory” may also be used herein to refer tonon-volatile mass storage such as physical storage media.

The computing system also has thereon multiple structures often referredto as an “executable component.” For instance, the memory of a computingsystem can include an executable component. The term “executablecomponent” is the name for a structure that is well understood to one ofordinary skill in the art in the field of computing as being a structurethat can be software, hardware, or a combination thereof.

For instance, when implemented in software, one of ordinary skill in theart would understand that the structure of an executable component mayinclude software objects, routines, methods, and so forth, that may beexecuted by one or more processors on the computing system, whether suchan executable component exists in the heap of a computing system, orwhether the executable component exists on computer-readable storagemedia. The structure of the executable component exists on acomputer-readable medium in such a form that it is operable, whenexecuted by one or more processors of the computing system, to cause thecomputing system to perform one or more functions, such as the functionsand methods described herein. Such a structure may be computer-readabledirectly by a processor—as is the case if the executable component werebinary. Alternatively, the structure may be structured to beinterpretable and/or compiled—whether in a single stage or in multiplestages—so as to generate such binary that is directly interpretable by aprocessor.

The term “executable component” is also well understood by one ofordinary skill as including structures that are implemented exclusivelyor near-exclusively in hardware logic components, such as within a fieldprogrammable gate array (FPGA), an application specific integratedcircuit (ASIC), Program-specific Standard Products (ASSPs),System-on-a-chip systems (SOCs), Complex Programmable Logic Devices(CPLDs), or any other specialized circuit. Accordingly, the term“executable component” is a term for a structure that is well understoodby those of ordinary skill in the art of computing, whether implementedin software, hardware, or a combination thereof.

The terms “component,” “service,” “engine,” “module,” “control,”“generator,” or the like may also be used in this description. As usedin this description and in this case, these terms whether expressed withor without a modifying clause—are also intended to be synonymous withthe term “executable component” and thus also have a structure that iswell understood by those of ordinary skill in the art of computing.

While not all computing systems require a user interface, in someembodiments a computing system includes a user interface for use incommunicating information from/to a user. The user interface may includeoutput mechanisms as well as input mechanisms. The principles describedherein are not limited to the precise output mechanisms or inputmechanisms as such will depend on the nature of the device. However,output mechanisms might include, for instance, speakers, displays,tactile output, projections, holograms, and so forth. Examples of inputmechanisms might include, for instance, microphones, touchscreens,projections, holograms, cameras, keyboards, stylus, mouse, or otherpointer input, sensors of any type, and so forth.

Accordingly, embodiments described herein may comprise or utilize aspecial purpose or general-purpose computing system. Embodimentsdescribed herein also include physical and other computer-readable mediafor carrying or storing computer-executable instructions and/or datastructures. Such computer-readable media can be any available media thatcan be accessed by a general purpose or special purpose computingsystem. Computer-readable media that store computer-executableinstructions are physical storage media. Computer-readable media thatcarry computer-executable instructions are transmission media. Thus, byway of example—not limitation—embodiments disclosed or envisioned hereincan comprise at least two distinctly different kinds ofcomputer-readable media: storage media and transmission media.

Computer-readable storage media include RAM, ROM, EEPROM, solid statedrives (“SSDs”), flash memory, phase-change memory (“PCM”), CD-ROM orother optical disk storage, magnetic disk storage or other magneticstorage devices, or any other physical and tangible storage medium thatcan be used to store desired program code in the form ofcomputer-executable instructions or data structures and that can beaccessed and executed by a general purpose or special purpose computingsystem to implement the disclosed functionality of the invention. Forexample, computer-executable instructions may be embodied on one or morecomputer-readable storage media to form a computer program product.

Transmission media can include a network and/or data links that can beused to carry desired program code in the form of computer-executableinstructions or data structures and that can be accessed and executed bya general purpose or special purpose computing system. Combinations ofthe above should also be included within the scope of computer-readablemedia.

Further, upon reaching various computing system components, program codein the form of computer-executable instructions or data structures canbe transferred automatically from transmission media to storage media(or vice versa). For example, computer-executable instructions or datastructures received over a network or data link can be buffered in RAMwithin a network interface module (e.g., a “NIC”) and then eventuallytransferred to computing system RAM and/or to less volatile storagemedia at a computing system. Thus, it should be understood that storagemedia can be included in computing system components that also—or evenprimarily—utilize transmission media.

Those skilled in the art will further appreciate that a computing systemmay also contain communication channels that allow the computing systemto communicate with other computing systems over, for example, anetwork. Accordingly, the methods described herein may be practiced innetwork computing environments with many types of computing systems andcomputing system configurations. The disclosed methods may also bepracticed in distributed system environments where local and/or remotecomputing systems, which are linked through a network (either byhardwired data links, wireless data links, or by a combination ofhardwired and wireless data links), both perform tasks. In a distributedsystem environment, the processing, memory, and/or storage capabilitymay be distributed as well.

Those skilled in the art will also appreciate that the disclosed methodsmay be practiced in a cloud computing environment. Cloud computingenvironments may be distributed, although this is not required. Whendistributed, cloud computing environments may be distributedinternationally within an organization and/or have components possessedacross multiple organizations. In this description and the followingclaims, “cloud computing” is defined as a model for enabling on-demandnetwork access to a shared pool of configurable computing resources(e.g., networks, servers, storage, applications, and services). Thedefinition of “cloud computing” is not limited to any of the othernumerous advantages that can be obtained from such a model when properlydeployed.

A cloud-computing model can be composed of various characteristics, suchas on-demand self-service, broad network access, resource pooling, rapidelasticity, measured service, and so forth. A cloud-computing model mayalso come in the form of various service models such as, for example,Software as a Service (“SaaS”), Platform as a Service (“PaaS”), andInfrastructure as a Service (“IaaS”). The cloud-computing model may alsobe deployed using different deployment models such as private cloud,community cloud, public cloud, hybrid cloud, and so forth.

Although the subject matter described herein is provided in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts so described.Rather, the described features and acts are disclosed as example formsof implementing the claims.

CONCLUSION

The present disclosure may be embodied in other specific forms withoutdeparting from its spirit or essential characteristics. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Whilecertain embodiments and details have been included herein and in theattached disclosure for purposes of illustrating embodiments of thepresent disclosure, it will be apparent to those skilled in the art thatvarious changes in the methods, products, devices, and apparatusesdisclosed herein may be made without departing from the scope of thedisclosure or of the invention. Thus, while various aspects andembodiments have been disclosed herein, other aspects and embodimentsare contemplated. All changes that come within the meaning and range ofequivalency of the claims are to be embraced within their scope.

EXAMPLES

The following examples as set forth herein are intended for illustrativepurposes only and are not intended to limit the scope of the disclosurein any way.

Example 1

An exemplary camera system used in a live clinical workflow helpedidentify two wrong-patient errors in the first 8,000 images obtained bythe camera system within the first seven months of deployment anddemonstrated numerous cases where point-of-care photography providedimage-related clinical context impacting diagnosis.

The camera system was supported by a Raspberry Pi Zero W controllerenclosed in a 3D printed case and mounted on a portable digital X-raymachine. The camera system incorporated a wide-angle lens and wastriggered by a pressure-sensitive mechanical switch that was mounted onthe X-ray machine hand-switch so that the patient photo was obtainedsynchronously with the radiograph. To enhance security, photos wereretrieved wirelessly from the camera by a storage silo (i.e., the PatCamServer (PS)) over a WPA2 secured wireless network hosted by the PS—thuseliminating any need for the camera to have credentials for logging intoor sending photos to the PS. Upon successful retrieval of the photos atthe PS, the copies local to the camera were deleted. The PS interactedwith the PACS server using a wired Ethernet connection, separate fromthe secured wireless network used for the aforementioned photoretrieval. All local data containing private health information wasautomatically deleted daily.

It was clear to the radiologists that the errors identified using thecamera system above would have been missed without the correspondingphotographs. For one of the misidentified studies, the patient photo wasused by a radiologist to confirm the error. Furthermore, the same photowas used by the X-ray technologists to find the correct patient. This isunprecedented since misfiled radiographs are usually discarded, implyingthat a misidentified patient underwent repeat imaging with its attendantradiation.

Example 2

In an observer study, radiologists interpreting portable radiographs,with and without the presence of synchronously obtained photographs,showed significantly increased confidence in interpretation withphotographs. The results also showed that feeding tubes and endotrachealtubes were more accurately identified on radiographs when the photoswere available during interpretation.

Example 3

Despite the Joint Commission mandated dual-identifier technique,wrong-patient errors still occur. In a retrospective conducted at EmoryHealthcare, 67 wrong-patient errors were identified over a 3.5-yearperiod. This search only identified errors that were discovered byradiologists after a report was generated and an addendum wassubsequently issued containing either the phrase “wrong patient” or thephrase “incorrect patient.” This is clearly the lower bound since manyerrors may have not been identified and those that were identified maynot have been issued addenda with such phrases.

Example 4

Two pilot studies with simulated wrong-patient errors were conducted atEmory University to validate the effect of patient photographs onradiologists' interpretation—one study with ten radiologists and asecond study with five radiologists. In a third study, 90 radiologistexaminers were asked to interpret ten pairs of portable chestradiographs either with or without the presence of photographs. The keyfinding of these three studies was that the presence of photographs ofthe patient (e.g., the patient's face and chest) at the time ofinterpretation of portable chest radiographs significantly increased thedetection rate of simulated wrong-patient errors. In the first study,with a set of 20 purportedly paired patient radiographs, theradiologists detected wrong-patient errors 12.5% of the time; afterintroduction of the photographs shown in tandem with the 20 pairs ofpurportedly paired patient radiographs, the error detection rateincreased to 64% (p=0.0003). Similarly, in the second study, thewrong-patient error detection rate increased from 0% to 94.4%(p<0.0001). In the larger study with 90 radiologists, the detection rateof simulated wrong patient errors increased from 31% to 77% (p=0.006)following the introduction of photographs.

These studies led to an intriguing and unexpected finding: showingphotographs of the patient led to a decreased time in interpretation ofthe radiograph. In the ten-radiologist study, the interpretation timefor 20 pairs of radiographs decreased from 35.73 to 26.51 minutes(p=0.1165), and in the five-radiologist study, it decreased from 26.45to 20.55 minutes (p=0.1911). These studies were designed to evaluatechanges in error detection rates and not to determine if interpretationtime would change. However, this unexpected finding was very intriguingand indicated that the interpretation time of radiographs may decreaseas a result of the interpreting radiologists being presented with noveland salient clinical information via the photographs.

What is claimed is:
 1. A method for identifying a disease state in apatient, comprising: compiling a dataset, the dataset comprising: aplurality of video recordings of a plurality of patients; medical imagedata associated with the plurality of patients; and one or morediagnoses or disease states associated with each unique subset of videorecordings and medical image data for the plurality of patients;generating a machine learning model based on the dataset; identifying aset of predictive characteristics associated with at least some of theone or more diagnoses or disease states based on the machine learningmodel; acquiring a video recording and medical image data associatedwith a particular patient; identifying characteristics within the videorecording and medical image data corresponding to one or more diseasestates; and determining a disease state in the particular patient basedon the identified characteristics, wherein determining the disease statein the patient comprises comparing the identified characteristics withinthe video recording and medical image data of the patient with the setof predictive characteristics; and selecting a probable disease statebased on the comparison.
 2. The method of claim 1, wherein the machinelearning model comprises a deep learning convolutional neural network.3. The method of claim 1, wherein each of the plurality of videorecordings of the plurality of patients or the video recordingassociated with the particular patient comprises an infrared video. 4.The method of claim 1, wherein one or more video frames of the videorecording associated with the particular patient include textual patientdata on a patient information board and/or patient monitor, the methodfurther comprising automatically extracting the textual patient datafrom the one or more video frames.
 5. The method of claim 1, furthercomprising comparing one or more video frames of the video recordingassociated with the particular patient and medical image data associatedwith the particular patient, and adjusting the one or more video framesrelative to the medical image data according to one or more of scaling asize of the one or more video frames, translating the one or more videoframes, rotating the one or more video frames, skewing the one or morevideo frames, or adjusting a brightness, contrast, color, saturation, oropacity of the one or more video frames.
 6. The method of claim 1,further comprising administering a therapeutic regimen to the patientbased on the determined disease state.
 7. The method of claim 1, furthercomprising automatically detecting laterality of one or more images ofthe medical image data based on one or more video frames of the videorecording associated with the particular patient.
 8. The method of claim7, further comprising comparing the detected laterality of the one ormore images of the medical image data to manually entered lateralitydata, and automatically flagging images of the one or more images havinga mismatch between detected laterality and manually entered laterality.9. A method for identifying a disease state in a patient, comprising:compiling a dataset, the dataset comprising: a plurality of infraredimages of a plurality of patients; medical image data associated withthe plurality of patients; and one or more diagnoses or disease statesassociated with each unique subset of infrared images and medical imagedata for the plurality of patients; generating a machine learning modelbased on the dataset; identifying a set of predictive characteristicsassociated with at least some of the one or more diagnoses or diseasestates based on the machine learning model; acquiring an infrared imageand medical image data associated with a particular patient; identifyingcharacteristics within the infrared image and medical image datacorresponding to one or more disease states; and determining a diseasestate in the particular patient based on the identified characteristics,wherein determining the disease state in the patient comprises comparingthe identified characteristics within the infrared image and medicalimage data of the patient with the set of predictive characteristics;and selecting a probable disease state based on the comparison.
 10. Themethod of claim 9, wherein the machine learning model comprises a deeplearning convolutional neural network.
 11. The method of claim 9,wherein each of the plurality of infrared images of the plurality ofpatients or the infrared image associated with the particular patientcomprises a thermal image.
 12. The method of claim 9, wherein each ofthe plurality of infrared images of the plurality of patients or theinfrared image associated with the particular patient comprises a partof a respective infrared video.
 13. The method of claim 9, furthercomprising comparing the infrared image and medical image dataassociated with the particular patient, and adjusting the infrared imagerelative to the medical image data according to one or more of scaling asize of the infrared image, translating the one or more video frames,rotating the infrared image, or adjusting a brightness, contrast, color,saturation, or opacity of the infrared image.
 14. The method of claim 9,further comprising administering a therapeutic regimen to the patientbased on the determined disease state.
 15. The method of claim 9,further comprising automatically detecting laterality of one or moreimages of the medical image data based on the infrared image associatedwith the particular patient.
 16. The method of claim 15, furthercomprising comparing the detected laterality of the one or more imagesof the medical image data to manually entered laterality data, andautomatically flagging images of the one or more images having amismatch between detected laterality and manually entered laterality.17. A method for identifying a disease state in a patient, comprising:compiling a dataset, the dataset comprising: a plurality of audiorecordings of a plurality of patients; medical image data associatedwith the plurality of patients; and one or more diagnoses or diseasestates associated with each unique subset of audio recordings andmedical image data for the plurality of patients; generating a machinelearning model based on the dataset; identifying a set of predictivecharacteristics associated with at least some of the one or morediagnoses or disease states based on the machine learning model;acquiring an audio recording and medical image data associated with aparticular patient; identifying characteristics within the audiorecording and medical image data corresponding to one or more diseasestates; and determining a disease state in the particular patient basedon the identified characteristics, wherein determining the disease statein the patient comprises comparing the identified characteristics withinthe audio recording and medical image data of the patient with the setof predictive characteristics; and selecting a probable disease statebased on the comparison.
 18. The method of claim 17, wherein the machinelearning model comprises a deep learning convolutional neural network.19. The method of claim 17, wherein the audio recording associated withthe particular patient includes one or more statements by the particularpatient.
 20. The method of claim 17, further comprising administering atherapeutic regimen to the patient based on the determined diseasestate.